Currently, there is a large amount of stored and accessible multimedia data including not only text information but also image information and audio information. In particular, accompanying the development of the internet, the amount of information on the WWW (World Wide Web) has been increasing steadily, so that there is a large amount of various kinds of stored and accessible multimedia data including the text information, the image information and the audio information. As a result, it has become increasingly difficult to retrieve necessary information accurately from the WWW.
Conventionally known information retrieval systems on the WWW and information arranging operations therein are described in the following.
First, a text key retrieval system is known. With respect to multimedia data to be a retrieval target, one or more pieces of text information representing their contents are assigned suitably. When retrieving the data, a text such as a word is specified as a retrieval key, whereby the text key retrieval system retrieves the multimedia data to which text information that is the same as this retrieval key has been assigned. In the case where the media information to be a retrieval target is text information, since the contents of the text information are retrieved by a text key, such a text key retrieval generally is carried out in an appropriate manner. In this text key retrieval system, the information can be arranged, for example, by sorting the data that have been hit in the retrieval and displaying a list thereof.
Second, a full-text retrieval system is known. This retrieval system is particularly effective when multimedia data to be a retrieval target are pieces of text information. There are several suggested methods therefor. For example, as a preprocessing, all the words representing features of the contents of the full text are extracted automatically from the full text, and a retrieval key file is generated so that these words are assigned thereto. When retrieving the data, a text such as a word is specified as a retrieval key, thereby detecting the text information whose retrieval key file contains a text that is the same as this retrieval key. With this method, like the first retrieval method using the text as a key, media information other than the text information, for example, the image information, is difficult to retrieve appropriately by using the text as a key, unless a suitable text key is assigned to each piece of the image information.
In this full-text retrieval system, the information also can be arranged, for example, by sorting the data that have been hit in the retrieval and displaying a list thereof, as in the text key retrieval system.
Third, as a method for retrieving the image information, further known is an image retrieval method of image pattern matching by specifying a part of an image and using the partial image as a retrieval key. With this image retrieval method, the image information having the specified partial image can be retrieved.
In this image retrieval system of image pattern matching, the information also can be arranged, for example, by sorting the data that have been hit in the retrieval and displaying a list thereof.
However, the conventional information retrieval systems on the WWW do not involve an effective retrieval method for the media information other than the text information, for example, the image information and the audio information.
With the firstly-described conventional text key retrieval system, it is difficult to retrieve the image information and the audio information appropriately. In other words, even if a plurality of keywords are assigned to one piece of image information, it is still difficult to carry out an appropriate and flexible image retrieval by the keywords in accordance with a searcher's intention because of the difficulty in representing the feature of the image accurately and flexibly with the text. Even when the retrieval is carried out indirectly using a text retrieval server or the like, the resultant information is not very useful. Although it is possible to collect a lot of images and display them sequentially, there has been a problem in that too many images would make it difficult to find an intended image.
With the secondly-described conventional full-text retrieval system, it is difficult to retrieve the image information and the audio information appropriately as in the first text key retrieval system. In other words, since the image information and the audio information originally do not have text information, no text information extraction can be expected therefrom.
With the thirdly-described conventional image retrieval system of image pattern matching, it is possible to retrieve the image information having a specified partial image. However, a searcher has to prepare and specify a partial image contained in the image that he/she wants to retrieve. This makes it difficult to retrieve the image information because what kind of partial image is contained in the image that he/she wants to retrieve the most is not clear in some cases and a partial image used for the retrieval cannot always be prepared. Moreover, a searcher sometimes does not know clearly the image that he/she wants to retrieve. In other words, there are some cases where a searcher can specify a general content of the image that he/she wants to retrieve but cannot specify the image itself even partially. There also are cases where a searcher just wants to carry out a trial-and-error retrieval to find out any usable image indeterminately. In such cases, matching only partial images is not flexible enough and, therefore, insufficient.